Markov Chain Monte Carlo and Numerical Differential Equations

This contribution presents a —hopefully readable— introduction to Markov Chain Monte Carlo methods with particular emphasis on their combination with ideas from deterministic or stochastic numerical differential equations. Markov Chain Monte Carlo algorithms are widely used in many sciences, including physics, chemistry and statistics; their importance is comparable to those of the Gaussian Elimination or the Fast Fourier Transform. We have tried to keep the presentation as self-contained as it has been feasible. A basic knowledge of applied mathematics and probability1 is assumed, but there are tutorial sections devoted to the necessary prerequisites in stochastic processes (Section 2), Markov chains (Section 3), stochastic differential equations (Section 6) and Hamiltonian dynamics/statistical physics (Section 8). The basic Random Walk Metropolis algorithm for discrete or continuous distributions is presented in Sections 4 and 5. Sections 7 and 9 are respectively devoted to MALA, an algorithm based on stochastic differential equations proposals, and to the Hybrid Monte Carlo method, founded on ideas from Hamiltonian mechanics. We have avoided throughout mathematical technicalities (that in the study of continuous-time stochastic processes may be overwhelming). We have rather followed the style of presentation taken by D. Higham in his tutorial paper on stochastic differential equations [18] and aimed at an exposition based on computer experiments; we believe that this approach may provide much insight and be a very useful entry point to the study of the issues considered here.